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ABSTRACT 



In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in NAEP's 
history, voluntary state-by-state assessments were made. In 1992, the NAEP 
Program included an expanded TSA in fourth-grade reading. The 1992 reading 
assessment considered students' performance in situations that involved 
reading different kinds of materials for different purposes. The fourth-grade 
assessment measured two global purposes for reading- -reading for literary 
experience and reading to gain information. In West Virginia, 2,733 students 
in 137 public schools were assessed. This report describes the reading 
proficiency of West Virginia fourth-graders and compares their overall 
performance to students in the Southeast region of the United States and the 
nation (using data from the NAEP national assessments) . The distribution of 
reading results and reading achievement level results are provided for 
subpopulations of students (race/ethnicity, type of location, parents' 
educational level, and gender, and performance according to purpose for 
reading) . To provide a context for the assessment data, participating public 
school students, their reading teachers, and principals completed 
questionnaires which focused on: policies and practices related to reading 
(time for instructional activities and instructional resources for reading) ; 
delivery of reading instruction (instructional materials and activities, 
workbooks/worksheets/writing, discussion and group activities, time to read, 
reading and use of libraries, assessing progress in reading) ; reading 
instructors (preparation, experience and professional development); students' 
home support for literacy (reading outside of school and in the home and 
hours of television watched per day) . The average reading proficiency of 
fourth-grade public school students in West Virginia on the NAEP reading 
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What is The Nation’s Report Card ? 



THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally representative and continuing assessment 
of what America’s students know and can do in various subject areas. Since 1969, assessments have been conducted periodically in reading, mathematics, 
science, writing, history/geography, and other fields. By making objective information on student performance available to policymakers at the national, 
state, and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information related to academic 
achievement is collected under this program. NAEP guarantees the privacy of individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of Education. The Commissioner of 
Education Statistics is responsible, by law, for carrying out the NAEP project through competitive awards to qualified organizations. NAEP reports directly 
to the Commissioner, who is also responsible for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP's 
conduct and usefulness. 

In 1988, Congress created the National Assessment Governing Board (NAGB) to formulate policy guidelines for NAEP. The board is responsible for 
selecting the subject areas to be assessed, which may include adding to those specified by Congress; identifying appropriate achievement goals for each age 
and grade; developing assessment objectives; developing test specifications; designing the assessment methodology; developing guidelines and standards 
for data analysis and for reporting and disseminating results; developing standards and procedures for interstate, regional, and national comparisons; improving 
the form and use of the National Assessment; and ensuring that all items selected for use in the National Assessment are free from racial, cultural, gender, 
or regional bias. 
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The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation’s only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientifically selected samples of youths 
attending both public and private schools and enrolled in grades four, eight, or twelve. The 
assessment questions are written around a framework prepared for each content area — 
reading, writing, mathematics, science, and others — that represents the consensus of groups 
of curriculum experts, educators, members of the general public, and user groups on what 
should be covered on such an assessment. Reporting includes means and distributions of 
scores, as well as more descriptive information about the meaning of the data. 



New Reading Assessment Framework and Questions 



The goal of the National Center for Education Statistics is to make data available for the 
public and to do so in accurate and understandable ways that are not misleading. The task 
is challenging because much of what matters in NAEP is changing: 



• the content in response to the developing standards of various curricular 
groups; 

• the assessment questions in response to new developments in assessments; 
and 

• the reporting in response to increasing interest in student achievement 
relative to standards of student performance. 
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The framework for the 1992 Trial State Assessment Program in reading considered 
students’ performance in situations that involved reading different kinds of materials for 
different purposes. The fourth- grade reading assessment measured two global purposes for 
reading — reading for literary experience and reading to gain information. (The eighth- and 
twelfth-grade national NAEP reading assessments also measured a third purpose for 
reading - reading to perform a task.) Reading for literary experience usually involves the 
reading of novels, short stories, plays, and essays. In these reading situations, the reader 
can determine how the author explores or uncovers experiences through the text and 
considers the interplay among events, emotions, and possibilities. Reading to gain 
information usually involves the reading of articles in magazines and newspapers, chapters 
in a textbook, entries in encyclopedias and catalogs, and entire books on particular topics. 
These reading situations call for different orientations to text from those in reading for 
literary experience because readers are specifically focused on acquiring information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

• Initial Understanding - comprehending the overall or general meaning of 
the selection. 

• Developing an Interpretation — extending the ideas in the text by making 
inferences and connections. 

• Personal Response — making explicit connections between ideas in the text 
and a student’s own background knowledge and experiences. 

• Critical Stance -- considering how the author crafted a text. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. 

The 1992 NAEP reading assessment uses a variety of innovative assessment approaches 
that are considered significant advancements over previous assessments. In addition to 
multiple-choice questions, the assessment primarily includes constructed-response 
questions that ask students to demonstrate comprehension beyond a surface level. Also, 
longer and naturally-occurring reading materials are used to provide more realistic reading 
experiences than in previous assessments. 

Taken together, the changes in the 1992 reading framework and assessment activities 
preclude any comparisons between the results in this report and those for previous NAEP 
reading assessments. If the current NAEP framework is used in the future, as planned in 
the 1994 assessment, the 1992 reading data will supply the basis for a trend report. 
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A Transition in Reporting 

Over time there have been many changes in emphasis of NAEP reporting, both to take 
advantage of new technologies and to reflect changing trends in education. In 1984, a new 
technology called Item Response Theory (IRT) made it possible to create “scale scores” 
for NAEP similar to those the public was accustomed to seeing for the annual Scholastic 
Aptitude Test (SAT). Educational Testing Service, in its role as Government grantee 
carrying out NAEP operations, devised a new way to describe performance against this 
scale, called “anchor levels.” Starting in 1984, NAEP results were reported by “anchor 
levels.” Anchor levels describe distributions of performance at selected points along the 
NAEP scale (i.e., standard deviation units). Anchor levels show how groups of students 
perform relative to each other, but not whether this performance is adequate. 

In 1988, Congress established the National Assessment Governing Board (NAGB), 
assigning it broad policy making authority over NAEP, including the authority to take 
“appropriate actions ... to improve the form and use of the National Assessment” and to 
identify “appropriate achievement goals for each . . . grade and subject area to be tested in 
the National Assessment.” To carry out its responsibilities, NAGB developed 
“achievement levels,” which are collective judgments about how students should perform 
relative to a body of content reflected in the NAEP frameworks. The result is translated 
onto ranges along the NAEP scale. For the 1992 reading assessment, this process was 
conducted for NAGB under contract by American College Testing (ACT), which has 
extensive experience in standard-setting in many fields. 

With this background, the initial reports for the 1992 reading assessment mark NCES’s 
continued attempt to shift to standards-based reporting of National Assessment statistics. 
The first transition to reporting NAEP results by achievement levels was for the NAEP 
1992 Trial State Assessment in mathematics. 1 The impetus for this transition lies in the 
belief that NAEP data will take on more meaning for the public if they show what 
proportion of our youth are able to meet judgmental standards of performance. 

Reporting NAEP results on the basis of achievement levels represents a significant change 
in practice for NCES. On occasion, this agency makes use of emerging analytical 
approaches that permit new, and sometimes controversial, analyses to be done. When 
doing so, this agency, just as other statistical agencies do when introducing new measures 
to supplement or replace old measures, also has provided the data according to the earlier 
procedures in addition to the new ones. In the case of the 1992 mathematics assessment, 
for example, the “anchor levels” or “scale anchoring” method of reporting was presented 
in an appendix. 



1 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993) and the individual 
1992 Mathematics State Reports. 
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In this assessment, the “scale anchoring” methodology used by NAEP since 1985 has been 
used but in a new way. As implemented for this report, the scale anchoring process applies 
not to regular scale intervals (standard deviation units), but to the achievement levels 
established for fourth-grade students. 2 The details of this procedure are presented in 
Appendix D. The critical distinction here is that setting achievement levels attempts to 
describe what students should be able to do in various ranges of the NAEP scale while the 
anchoring procedure attempts to describe what they can do at those achievement levels 
using actual student performance data from the NAEP assessments. 

Chapter 1 of this report describes how the 1992 standards were prepared and provides 
examples of assessment questions that illustrate the reading content reflected in the 
descriptions of the NAEP achievement levels. Chapters 1 - 6 include information on 
overall means, distributions of reading proficiency, as well as background questionnaire 
data, all taken directly from the results of the assessment questions. 

Continuing Development Effort 

We believe that the numerous completed and ongoing studies 3 will lead to national debate 
that can assure the public is well informed about these issues — as informed they must be 
because the results will be a vital influence on what Americans come to think about the 
condition and progress of our schools. Indeed, measures of student learning may be as 
significant bases for public understanding about our nation’s education system as the 
Consumer Price Index and the monthly unemployment statistics are in informing the 
public about our nation’s economy. 

In addition, members of the public need the data in this report to see for themselves what 
standards-based reporting might do and to evaluate the often conflicting claims of adherents 
and detractors of these changes in approaches to reporting on the educational achievement 
of American students. Reporting NAEP results to the public would be more clear if the 
language of the achievement levels, or standards, could also directly describe what students 
know and can do. In order to accomplish that, the frameworks, assessment questions, and 
achievement levels may need to be developed in tandem. That is easier to say than to do, 
however, because it implies a substantially larger pool of assessment questions, carefully 
designed to support reporting about performance relative to a set of performance standards. 
Clearly this is a developmental effort that will take time and several iterations, during which 
data supporting appropriate inferences about the performance of American students will 
be gathered on a continuing basis. 



2 First, students were identified who performed at or around the three achievement levels on the scale (212, 243, 
and 275). Next, questions were identified that were answered correctly by 65 percent or more of the 
fourth-grade students at the cutpoint for that achievement level. Finally, reading educators were asked to 
analyze each anchor-level question and create summary descriptions of the skills and abilities evidenced by 
students who answered these sets of questions successfully. 

3 Educational Achievement Standards: Setting Achievement Levels for the Nation. The Second Report of the 
National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1992 Trial 
State Assessment. (Stanford, CA: National Academy of Education, 1993).; U.S. General Accounting Office 
(GAO). Educational Achievement Standards: NAGB's Approach Yields Misleading Interpretations. June 
1993. GAO/PEMD-93-12.; Assessing Student Achievement in the States. The First Report of the National 
Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1990 Trial State 
Assessment. (Stanford, CA: National Academy of Education, 1992).; R.L. Linn, D.M. Koretz, E.L. Baker, 
and L. Burstein. The Validity and Credibility of the Achievement Levels for the 1990 National Assessment of 
Educational Progress in Mathematics. (Los Angeles, CA: Center for Research on Evaluation, Standards, and 
Student Testing, UCLA, June 1991) CRESST Report 330. 
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SUMMARY 



A Recent History of NAEP 

In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by- state assessments on a trial basis. 

As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 4 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 5 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 

School and Student Participation in the Reading Assessment 

In West Virginia, 137 public schools participated in the fourth-grade reading assessment. 
The weighted school participation rate was 100 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 100 percent of all the 
fourth-grade public-school students in West Virginia. 



4 For a summary of the 1990 program, see lna V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement: NAEP’s 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

5 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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In total, 2,733 fourth-grade West Virginia public-school students were assessed. The 
weighted student participation rate was 96 percent. This means that the sample of students 
who took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in West Virginia 
(that is, all students from the population represented by the participating schools, minus 
those students excluded from the assessment). 



The overall weighted response rate (school rate times student rate) was 96 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 96 percent of the eligible fourth-grade public-school student population 
in West Virginia. 

Students’ Reading Performance 



As shown in the following figure, the overall average proficiency of fourth-grade 
public-school students from West Virginia on the NAEP reading scale was 217. This 
proficiency was about the same as that of students across the nation (2 16). 6 There also 
was a tremendous range in student performance. The lowest performing 10 percent of the 
fourth graders from West Virginia had proficiency levels below 172 while the top 10 percent 
of the fourth graders had proficiency levels above 258. 

THE I 
REPORT 

Fourth-Grade Public-School Students’ CARD 

Average Reading Proficiency ^ 

Trial State Assessment 





The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 



6 Differences reported as significant are statistically different at the 95 percent confidence level. This means that 
with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. 
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LEVELS OF ACHIEVEMENT 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment." (Pub. L. 297-100 Section 3403 (a)(5)(B)(ii)). 

NAGB developed three achievement levels for each grade — Basic, Proficient, and 
Advanced. Performance at the Basic level denotes partial mastery of the knowledge and 
skills that are fundamental for proficient work at each grade level. The central level, called 
Proficient, represents solid academic performance at each grade level tested. Students 
reaching this level demonstrate competency over challenging subject matter and are well 
prepared for the next level of schooling. Achievement at the Advanced level signifies 
superior performance at the grade tested. Definitions of the three levels of reading 
achievement are given below. 



BASIC i j 
1 LEVEL . : 


Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read. When reading texts 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences. 


if- 

■ «r ' 

JL. -Jfc-:*.: i 


Fourth-grade students performing at the Proficient level should be able to 
demonstrate an overall understanding of the text , providing inferential as well 
as literal information. When reading text appropriate to fourth grade, they 
should be able to extend the ideas in the text by making inferences , drawing 
conclusions , and making connections to their own experiences. The connection 
between the text and what the student infers shouid be clear. 


•^AEJVANCBP-: 
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Fourth-grade students performing at the Advanced level shouid be able to 
generalize about topics in the reading selection and demonstrate an awareness 
of how authors compose and use literary devices. When reading text 
appropriate to fourth grade, they shouid be able to judge texts critically and, in 
general, give thorough answers that indicate careful thought. 



Because the process of setting the levels of reading achievement centered on the descriptions 
of what students should be able to do, it is important to explore whether students actually 
met the expectations for performance at the Basic, Proficient, and Advanced levels. To 
help in this process, NCES arranged for ETS to apply a modified anchoring procedure to 
the 1992 reading achievement levels. A committee of reading education experts was 
assembled to review the questions and assessment results. Using their knowledge of reading 
and student performance on the individual questions, the committee members were asked 
to summarize student performance at each achievement level (see Appendix D for more 
details on the anchoring procedure). 

Placing the descriptions of how students performed at each of the levels in the context of 
the expectations for achievement at each of the levels and cross-checking with the actual 
question-by-question results yields some interesting findings. In general, the sets of reading 
skills expected were those observed. However, in some instances, particularly for extended 
response questions, even Advanced-level students had difficulty providing in-depth answers. 
In some other instances, because the assessment was developed prior to the achievement 
level descriptions, particular reading skills were not measured. For more information about 
student performance, see the full report. 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



16 



7 



West Virginia 



The following figure provides the percentage of students at or above each achievement 
level, as well as the percentage of students below the Basic level. In West Virginia, 

58 percent of the fourth graders in public schools were at or above the Basic level, 

22 percent were at or above the Proficient level, and 3 percent were at or above the 
Advanced level. Nationwide, 57 percent of the fourth graders were at or above the Basic 
level, 24 percent were at or above the Proficient level, and 4 percent were at or above the 
Advanced level. About the same percentage of students in West Virginia as across the 
nation were at or above the Proficient level. 
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The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by HH), If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 
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PERFORMANCE ACCORDING TO PURPOSE FOR READING 

The 1992 Trial State Assessment Program considered students’ performance in situations 
that involved reading different kinds of materials for different purposes. The fourth-grade 
reading assessment measured two global purposes for reading — reading for literary 
experience and reading to gain information. Students in West Virginia performed about the 
same as students across the nation in reading for literary experience and to gain 
information. 



Subpopulation Performance 



Assessment results repeatedly show differences in achievement for subpopulations of 
students. 7 The 1992 Trial State Assessment provides additional information about the 
achievement of important subpopulations by reporting on the performance of various 
subgroups of the student population defined by race/ethnicity, type of community, parents’ 
education level, and gender. 



RacelEthniclty 



White students in West Virginia demonstrated about the same average reading 
proficiency as did Black students and higher reading proficiency than did Hispanic 
students. In West Virginia, about one quarter of the White students (23 percent), 
relatively few of the Black students (10 percent), and some of the Hispanic 
students (13 percent) were at or above the Proficient level. Across the nation, 
about one quarter of the White students (30 percent), relatively few of the Black 
students (7 percent), and some of the Hispanic students (12 percent) were at or 
above the Proficient level. 



Type of Community 



The average reading performance of West Virginia students attending schools in 
areas classified as “other” was about the same as that of students attending schools 
in disadvantaged urban areas or extreme rural areas. About one quarter of the 
students attending schools in areas classified as “other” (22 percent), some of the 
students in disadvantaged urban areas (20 percent), and about one quarter of the 
students in extreme rural areas (23 percent) in West Virginia were at or above the 
Proficient level. Across the nation, about one quarter of the students in areas 
classified as “other” (24 percent), relatively few of the students in disadvantaged 
urban areas (5 percent), and about one quarter of the students in extreme rural 
areas (24 percent) were at or above the Proficient level. 



7 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress. (Washington, DC: National Center for Education Statistics, 1992). 
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Parents’ Education 



Students in West Virginia who reported that at least one parent graduated from 
college demonstrated about the same average reading proficiency as did students 
who reported that at least one parent had some education after high school, but 
higher proficiency than did students who reported that at least one parent 
graduated from high school, neither parent graduated from high school, or they 
did not know their parents’ education level. Reading achievement in West Virginia 
was at or above the Proficient level for 32 percent of the students who reported 
that at least one parent graduated from college, 29 percent of the students who 
reported that at least one parent had some education after high school, 18 percent 
of the students who reported that at least one parent graduated from high school, 
10 percent of the students who reported that neither parent graduated from high 
school, and 13 percent of the students who reported that they did not know their 
parents’ education level. Across the nation, these figures were 33 percent of the 
students who reported that at least one parent graduated from college, 28 percent 
of the students who reported that at least one parent had some education after high 
school, 18 percent of the students who reported that at least one parent graduated 
from high school, 10 percent of the students who reported that neither parent 
graduated from high school, and 17 percent of the students who reported that they 
did not know their parents’ education level. 



Gender | I n West Virginia, fourth-grade boys attending public schools had a lower average 
reading proficiency than did fourth-grade girls. Compared to the national results, 
girls in West Virginia performed about the same as girls across the country; boys 
in West Virginia performed about the same as boys across the country. There was 
a significant difference between the percentages of males and females in West 
Virginia who attained the Proficient level (26 percent for females and 18 percent 
for males). The percentage of females in West Virginia who attained the Proficient 
level was about the same as the percentage of females in the nation who attained 
the Proficient level (26 percent for West Virginia and 26 percent for the nation). 
Similarly, the percentage of males in West Virginia who attained the Proficient 
level was about the same as the percentage of males in the nation who attained the 
Proficient level (18 percent for West Virginia and 21 percent for the nation). 



A Context for Understanding Students’ Reading Proficiency 

Information on the reading performance of students in West Virginia can be better 
understood and used for improving instruction and setting policy when supplemented with 
contextual information about schools, teachers, and students. 

To gather contextual information, the fourth-grade students participating in the 1992 Trial 
State Assessment, their reading teachers, and the principals or other administrators in their 
schools were asked to complete questionnaires on policies, instruction, and programs. The 
student, teacher, and school data help to describe some of the current practices and 
emphases in reading education, illuminate some of the factors that appear to be related to 
fourth-grade public-school students’ reading proficiency, and provide an educational 
context for understanding information on student achievement. Highlights of the results 
for the public-school students in West Virginia are as follows: 
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CURRICULUM COVERAGE AND INSTRUCTIONAL EMPHASIS 

• In West Virginia, 83 percent of the fourth-grade students had reading 
teachers who spent at least 60 minutes providing reading instruction each 
day. By comparison, 17 percent of the students had reading teachers who 
spent 45 minutes or less providing reading instruction each day. 

• In West Virginia, some of the fourth-grade students (16 percent) were 
being taught by teachers who placed heavy emphasis on phonics; less than 
half (36 percent) were being taught by teachers who placed heavy emphasis 
on the integration of reading and writing; and about one quarter 
(26 percent) were being taught by teachers who placed heavy emphasis on 
the whole language approach. 

• In addition, in West Virginia, about one quarter of the fourth-grade 
students (24 percent) were being taught by teachers who placed heavy 
emphasis on literature-based reading; less than half (42 percent) were being 
taught by teachers who placed heavy emphasis on reading across the 
content areas; and relatively few (7 percent) were being taught by teachers 
who placed heavy emphasis on individualized reading programs. 



DELIVERY OF READING INSTRUCTION 

• About one quarter of the fourth-grade public-school students in West 
Virginia (25 percent) had reading teachers who used both basal and trade 
books, about three quarters (73 percent) had reading teachers who 
primarily used basal readers, and relatively few (1 percent) had reading 
teachers who primarily used trade books. 

• In West Virginia, 2 percent of the fourth-grade students had reading 
teachers who used children’s newspapers and/or magazines almost every 
day; 10 percent of the students had reading teachers who used reading kits 
almost every day; 6 percent had reading teachers who used computer 
software for reading instruction almost every day; 21 percent had reading 
teachers who used a variety of books almost every day; and, finally, 
23 percent had teachers who used materials from other subject areas 
almost every day. 

• In West Virginia, about one quarter of the fourth-grade students 

(26 percent) had reading teachers who devoted almost all of their 

instructional time in reading to teaching decoding skills; about one quarter 
of the students (27 percent) had reading teachers who devoted almost all 
of their instructional time in reading to oral reading; less than half 
(39 percent) had reading teachers who devoted almost all of their 

instructional time in reading to teaching vocabulary; more than half 
(57 percent) had reading teachers who devoted almost all of their 

instructional time in reading to comprehension/interpretation; and finally, 
less than half (33 percent) had teachers who devoted almost all of their 
instructional time in reading to reading strategies. 
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EDUCATIONAL BACKGROUND OF TEACHERS 

• In West Virginia, 56 percent of the students were being taught by reading 
teachers who reported having at least a master’s or education specialist’s 
degree. This compares with 46 percent for students across the nation. 

• More than half of the students (59 percent) had reading teachers who had 
the highest level of teaching certification that is recognized by West 
Virginia. Hus is about the same as the figure for the nation, where more 
than half of the students (57 percent) were taught by reading teachers who 
were certified at the highest level available in their states. 

• In West Virginia, 23 percent of the fourth-grade public-school students 
were being taught reading by teachers who had an undergraduate major in 
English, reading, and/or language arts. By comparison, 22 percent of the 
students across the nation had reading teachers with the same major. 

HOME FACTORS 

• Students in West Virginia who had four types of reading materials in the 
home (newspapers, magazines, more than 25 books, and an encyclopedia) 
showed a higher reading proficiency than did students with zero to two 
types of materials. Across the nation, students who had all four types of 
materials at home showed a higher reading proficiency than did students 
who had zero to two types. 

• In West Virginia, 27 percent of the fourth-grade public-school students 
discussed what they read with friends or family almost every day; 
24 percent never or hardly ever discussed what they read. Across the 
nation, 27 percent discussed what they read with friends or family almost 
every day and 24 percent never or hardly ever discussed what they read. 

• Some of the fourth-grade public-school students in West Virginia 
(15 percent) watched one hour or less of television each day; about one 
quarter (22 percent) watched six hours or more. 



Comparisons of Overall Reading Proficiency in West Virginia with 
Other States 

The map on the following page provides a method for making appropriate comparisons 
of the overall reading proficiency in West Virginia with that in other states (including the 
District of Columbia and one territory) that participated in the NAEP 1992 Trial State 
Assessment Program. The different shadings of the states on the map show whether the 
average overall proficiency in the other states was statistically different from or not 
statistically different from that in West Virginia (“Target State”). States with a dark-colored 
shading have a significantly higher average proficiency than does West Virginia. States with 
a light-colored shading have a significantly lower average proficiency than does West 
Virginia. States without shading have an average proficiency that does not differ 
significantly from that of West Virginia. The significance tests are based on a Bonferroni 
procedure for multiple comparisons that holds the probability of erroneously declaring the 
means of any two states to be different, when they are not, to no more than five percent. 
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In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis: 

The National Assessment shall develop a trial mathematics assessment survey 
instrument for the eighth grade and shall conduct a demonstration of the 
instrument in 1990 in States which wish to participate , with the purpose of 
determining whether such an assessment yields valid \ reliable State representative 
data . (Section 406(i)(2)(C)(i) of the General Education Provisions Act, as 
amended by Pub . L. 100-297 (U.S.C. 1221e-l(i)(2)(c)(i))) 

The National Assessment shall conduct a trial mathematics assessment for the 
fourth and eighth grades in 1992 and, pursuant to subparagraph (6)(D), shall 
develop a trial reading assessment to be administered in 1992 for the fourth grade 
in States which wish to participate, with the purpose of determining whether such 
an assessment yields valid, reliable State representative data . (Section 

406(i)(2)(C)(i) of the General Education Provisions Act, as amended by Pub . 

L. 100-297 (U.S.C. 1221e-l(i)(2)(c)(ii))) 



As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 8 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 9 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 




8 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement: NAEP's 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

9 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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The 1992 Trial State Assessment Program was conducted in February 1992 with the 
following 44 participants: 
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* The Virgin Islands participated in the testing portion of the 1992 Trial State Assessment Program. However, 
in accordance with the legislation providing for participants to review and give permission for release of their 
results, the Virgin Islands chose not to release their results at grade 4 in the reports. 



States in regular type did not participate in the 1990 Trial State Assessment. Three states 
~ Montana, Illinois, and Oregon — participated in the 1990 Trial State Assessment but not 
in the 1992 program. 

For the 1992 Trial State Assessment in reading, approximately 2,500 students were assessed 
in each jurisdiction. The samples were carefully designed to represent the fourth-grade 
public-school populations in the states or territories. Similar to the 1990 program, local 
school district personnel administered all assessment sessions, and the contractor’s staff 
monitored 50 percent of the sessions as part of the quality assurance program designed to 
ensure that the sessions were conducted uniformly. The results of the monitoring in 1990 
and 1992 indicated a high degree of quality and uniformity across sessions. 

The 1992 Trial State and National Assessment programs in reading were based on a 
framework developed through a national consensus process that was set forth by law and 
called for “active participation of teachers, curriculum specialists, subject matter specialists, 
local school administrators, parents, and members of the general public” (Public Law 
100-297, Part C, 1988). 10 



10 Reading Framework for the 1992 National Assessment of Educational Progress . (Washington, DC: National 
Assessment Governing Board, U.S. Department of Education, 1992). 
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The process of developing the framework was carried out in late 1989 and early 1990 by 
the Council of Chief State School Officers (CCSSO) under contract from the National 
Assessment Governing Board (NAGB) which is responsible for formulating policy for 
NAEP, including developing assessment objectives and test specifications. The framework 
development process included input from a wide range of people in the fields of reading 
and assessment, from school teachers and administrators to state coordinators of reading 
and reading assessment. After thorough discussion and some amendment, the framework 
was adopted by NAGB in March 1990. An overview of the reading framework is provided 
in the Procedural Appendix. 

The fourth-grade Trial State and National Assessments in reading included eight sections 
or blocks, each 25 minutes in length. Each block consisted of a passage and a combination 
of constructed-response and multiple -choice questions. Passages selected for the 
assessment were drawn from authentic texts used by students in real, everyday reading. 
Whole stories, articles, or sections of textbooks were used, rather than excerpts or 
abridgements. The type of question — constructed-response or multiple-choice -- was 
determined by the nature of the task. In addition, the constructed-response questions were 
of two types: regular constructed-response questions required students to respond to a 
question in a few words or a few sentences while extended constructed-response questions 
required students to respond to a question in a paragraph or more. 



This Report 



This is a computer-generated report that describes the reading performance of fourth-grade 
public-school students in West Virginia, in the Southeast region, and across the nation. 

A separate report describes additional fourth-grade reading assessment results for the nation 
and the states, as well as the national results for grades 8 and 12. 11 This report consists 
of three sections: 



• This Overview provides background information about the Trial State 
Assessment and a profile of the fourth-grade public-school students in West 
Virginia. 

• Part One describes the reading performance of the fourth-grade 
public-school students in West Virginia, the Southeast region, and the 
nation. 

• Part Two relates fourth-grade students’ reading performance to contextual 
information about the reading policies, instruction, and home support for 
reading in West Virginia, the Southeast region, and the nation. 



11 See NAEP 1992 Reading Report Card for the Nation and the States . (Washington, DC: National Center for 
Education Statistics, 1993). 
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In this report, results are provided for groups of students defined by shared characteristics 
— race/ethnicity, type of community, parents’ education level, and gender. Definitions of 
the subpopulations referred to in this report are presented below. The results for West 
Virginia are based on the representative sample of public-school students who participated 
in the 1992 Trial State Assessment Program. The results for the nation and the region of 
the country are based on the nationally and regionally representative samples of 
public-school students who were assessed in January through March as part of the national 
NAEP program. Using the regional and national results from the 1992 national NAEP 
program is necessary because of the voluntary nature of the Trial State Assessment 
Program. Since not every state participated in the program, the aggregated data across 
states did not necessarily provide representative national or regional results. Specific details 
on the samples and analysis procedures used can be found in the Technical Report of the 
1992 NAEP Trial State Assessment Program in Reading} 2 



RACE/ETHNICITY 

Results are presented for students of different racial/ethnic groups based on the students’ 
self-identification of their race/ethnicity according to the following mutually exclusive 
categories: White, Black, Hispanic, Asian (including Pacific Islander), and American 
Indian (including Alaskan Native). Based on criteria described in the Procedural Appendix, 
there must be at least 62 students in a particular subpopulation in order for the results for 
that subpopulation to be considered reliable. Thus, results for racial/ethnic groups with 
fewer than 62 students are not reported. However, the data for all students, regardless of 
whether their racial/ethnic group was reported separately, were included in computing 
overall results for West Virginia. 



TYPE OF COMMUNITY 

Results are provided for four mutually exclusive community types — advantaged urban, 
disadvantaged urban, extreme rural, and other — as defined below: 

Advantaged Urban: Students in this group live in metropolitan statistical areas 
and attend schools where, according to their schools, a high proportion of the 
students’ parents are in professional or managerial positions. 

Disadvantaged Urban : Students in this group live in metropolitan statistical 
areas and attend schools where, according to their schools, a high proportion 
of the students’ parents are on welfare or are not regularly employed. 



12 Technical Report of the NAEP 1992 Trial State Assessment Program in Reading. (Washington, DC: 
National Center for Education Statistics, 1993). 
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Extreme Rural: Students in this group live outside metropolitan statistical 

areas, live in areas with a population below 10,000, and attend schools where, 
according to their schools, many of the students’ parents are farmers or farm 
workers. 

Other: Students in this category attend schools in areas other than those defined 
as advantaged urban, disadvantaged urban, or extreme rural. 



Indices were developed such that approximately 10 percent of the most extreme advantaged 
urban, disadvantaged urban, and rural schools sampled in the national assessment were 
classified into these three categories. The remaining 70 percent of the schools were 
classified into the “other” category. The reporting of results by each type of community 
was also subject to a minimum student sample size of 62. 



PARENTS’ EDUCATION LEVEL 

Students were asked to indicate the extent of schooling for each of their parents — did not 
finish high school, graduated from high school, some education after high school, or 
graduated from college. The response indicating the higher level of education was selected 
for reporting. Reporting of results by parents’ education level was also subject to a 
minimum student sample size of 62. Note that a substantial percentage of fourth-grade 
students did not know their parents’ education level. 



GENDER 

Results are reported separately for males and females. 



REGION 

The United States has been divided into four regions for purposes of this report: Northeast, 
Southeast, Central, and West. States included in each region are shown in Figure 1. All 
50 states and the District of Columbia are listed, with the participants in the Trial State 
Assessment highlighted in boldface type. Territories were not assigned to a region. 
Further, the part of Virginia that is included in the Washington, DC, metropolitan 
statistical area is included in the Northeast region; the remainder of the state is included in 
the Southeast region. Because most of the Virginia students are in the Southeast region, 
regional comparisons for Virginia are to the Southeast. 

The regional results are based on a separate sample from that used to report the state 
results. Regional results are based on national assessment samples, not on aggregated Trial 
State Assessment samples. 
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Guidelines for Analysis 

This report describes reading proficiency for fourth graders attending public schools and 
compares the results for various groups of students within that population — for example, 
those who have certain demographic characteristics or who responded to a specific 
background question in a particular way. The report examines the results for individual 
groups and individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 

Because the proportions of students in these subpopulations and their average proficiencies 
are based on samples — rather than the entire population of fourth graders in public schools 
in a state or territory — the numbers reported are necessarily estimates. As such, they are 
subject to a measure of uncertainty, reflected in the standard error of the estimate. When 
the proportions or average proficiencies of certain groups are compared, it is essential to 
take the standard error into account, rather than rely solely on observed similarities or 
differences. Therefore, the comparisons discussed in this report are based on statistical tests 
that consider both the magnitude of the difference between the means or proportions and 
the standard errors of those statistics. 
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The statistical tests determine whether the evidence — based on the data from the groups 
in the sample - is strong enough to conclude that the means or proportions are really 
different for those groups in the population . If the evidence is strong (i.e., the difference is 
statistically significant ), the report describes the group means or proportions as being 
different (e.g., one group performed higher than or lower than another group) — regardless 
of whether the sample means or sample proportions appear to be about the same or not. 
If the evidence is not sufficiently strong (i.e., the difference is not statistically significant), 
the means or proportions are described as being about the same - again, regardless of 
whether the sample means or sample proportions appear to be about the same or widely 
discrepant. The reader is cautioned to rely on the results of the statistical tests — rather than 
on the apparent magnitude of the difference between sample means or proportions — to 
determine whether those sample differences are likely to represent actual differences 
between the groups in the population. The statistical tests and Bonferroni procedure, 
which is used when more than two groups are being compared, are discussed in greater 
detail in the Procedural Appendix. 

In addition, some of the percentages reported in the text of the report are given quantitative 
descriptions (e.g., some, about half, almost all, etc.). The descriptive phrases used and the 
rules used to select them are described in the Procedural Appendix. 

Finally, in several places in this report, results (mean proficiencies and proportions) are 
reported in the text for combined groups of students. For example, in the text, the 
percentage of students in the combined group where teachers reported spending 60 minutes 
or 90 minutes or more on reading instruction on a typical day is given and compared to the 
group where teachers reported spending 45 minutes or less. However, the table that 
accompanies that text reports percentages and proficiencies separately for the three groups 
(45 minutes or less, 60 minutes, and 90 minutes or more). The combined group 
percentages reported in the text and used in all statistical tests are based on unrounded 
estimates (i.e., estimates calculated to several decimal places) of the percentages in each 
group. The percentages shown in the tables are rounded to integers. Thus, percentages 
may not always add up to 100 percent due to rounding. Also, the percentage for a 
combined group (reported in the text) may differ slightly from the sum of the separate 
percentages (presented in the tables) for each of the groups that were combined. Therefore, 
if statistical tests were to be conducted based on the rounded numbers in the tables, the 
results might not be consonant with the results of the statistical tests that are reported in 
the text (based on unrounded numbers.) 

Profile of West Virginia 

FOURTH-GRADE SCHOOL AND STUDENT CHARACTERISTICS 

Table 1 provides a profile of the demographic characteristics of the fourth-grade 
public-school students in West Virginia, the Southeast region, and the nation. This profile 
is based on data collected from the students and schools participating in the 1992 Trial 
State and National Assessments. As described earlier, the state data and the regional and 
national data are drawn from separate samples. 
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TABLE 1 



Profile of Fourth-Grade 
Public-School Students in 
West Virginia, the Southeast 
Region, and the Nation 



West Virginia 



Southeast 



Nation 




DEMOGRAPHIC SUBGROUPS 



Race/Ethnicity 

White 

Black 

Hispanic 

Asian 

American Indian 

Type of Community 

Advantaged urban 
Disadvantaged urban 
Extreme rural 
Other 

Parents’ Education 

Graduated college 
Some education after high school 
Graduated high school 
Did not finish high school 
I don't know 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). The percentages for Race/ Ethnicity may not add to 100 percent because some 
students categorized themselves as “Other.” 



SCHOOLS AND STUDENTS ASSESSED 

Table 2 summarizes participation data for West Virginia schools and students sampled for 
the 1992 Trial State Assessment. 1 3 In West Virginia, 137 public schools participated in the 
fourth-grade reading assessment. These numbers include participating substitute schools 
that were selected to replace some of the nonparticipating schools from the original sample. 
The weighted school participation rate was 100 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 100 percent of all the 
fourth-grade public-school students in West Virginia. 



13 For a detailed discussion of the NCES guidelines for sample participation, see School and Student 
Participation Rates for the Reading Assessment . (Washington, DC: National Center for Education Statistics, 
1993); or see Appendix B of the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading . (Washington, DC: National Center for Education Statistics, 1993). 
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In each school, a random sample of students was selected to participate in the assessment. 
As estimated by the sample, 0 percent of the fourth-grade public-school population was 
classified as Limited English Proficient (LEP), while 8 percent in fourth grade had an 
Individualized Education Plan (IEP). An IEP is a plan, written for a student who has been 
determined to be eligible for special education, that typically sets forth goals and objectives 
for the student and describes a program of activities and/or related services necessary to 
achieve the goals and objectives. Handicapped or disabled students may be categorized 
as IEP. 

Schools were permitted to exclude certain students from the assessment, provided that 
certain criteria were met. To be excluded, a student had to be categorized as Limited 
English Proficient or had to have an Individualized Education Plan and (in either case) be 
judged incapable of participating in the assessment. The intent was to assess all selected 
students; therefore, all selected students who were capable of participating in the assessment 
should have been assessed. However, schools were allowed to exclude those students who, 
in the judgment of school staff, could not meaningfully participate. The NAEP guidelines 
for exclusion are intended to assure uniformity of exclusion criteria from school to school. 
Note that some LEP and IEP students were deemed eligible to participate and not excluded 
from the assessment. The students in West Virginia who were excluded from the 
assessment because they were categorized as LEP or had an IEP represented 5 percent of 
the population in grade four. 

In total, 2,733 fourth-grade West Virginia public-school students were assessed. The 
weighted student participation rate was 96 percent. This means that the sample of students 
who took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in West Virginia 
(that is, all students from the population represented by the participating schools, minus 
those students excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 96 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 96 percent of the eligible fourth-grade public-school population in West 
Virginia. 

In the analysis of student data and reporting of results, nonresponse weighting adjustments 
have been made at both the school and student level, with the aim of making the sample 
of participating students as representative as possible of the entire eligible fourth-grade 
public-school population. For details of the nonresponse weighting adjustment 
procedures, see the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading. 
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TABLE 2 



Profile of the Fourth-Grade 
Population Assessed in West 
Virginia 



PUBLIC SCHOOL PARTICIPATION 



Weighted school participation rate before substitution 

Weighted school participation rate after substitution 

Number of schools originally sampled 

Number of schools not eligible 

Number of schools in original sample participating 

Number of substitute schools provided 

Number of substitute schools participating 

Total number of participating schools 



PUBLIC-SCHOOL STUDENT PARTICIPATION 



Weighted student participation rate after makeups 

Number of students selected to participate in the assessment 

Number of students withdrawn from the assessment 

Percentage of students who were of Limited English Proficiency 

Percentage of students excluded from the assessment due to Limited 
English Proficiency 

Percentage of students who had an Individualized Education Plan 

Percentage of students excluded from the assessment due to Individualized 
Education Plan Status 

Number of students to be assessed 

Number of students assessed 

Overall weighted response rate 
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PART ONE 

How Proficient in Reading are Fourth-Grade 
Students in West Virginia Public Schools? 



EPORT 
CARD 

1992 

Trial State Assessment 




Reading involves the interaction between a reader, a text, and a situation. 14 Thus, 
students’ reading comprehension is influenced by the type of material read and the specific 
purposes for reading. The 1992 Trial State Assessment Program considered students’ 
performance in situations that involved reading different kinds of materials for different 
purposes. The fourth-grade reading assessment measured two global purposes for reading 
— reading for literary experience and reading to gain information . 15 Reading for literary 
experience usually involves the reading of novels, short stories, plays, and essays. In these 
reading situations, the reader can determine how the author explores experiences through 
the text and can consider the interplay among events, emotions, and possibilities. Reading 
to gain information usually involves the reading of articles in magazines and newspapers, 
chapters in a textbook, entries in encyclopedias and catalogs, and entire books on particular 
topics. These reading situations call for different orientations to text from those in reading 
for literary experience because readers are specifically focused on acquiring information. 
Students’ performance on each of the two purposes for reading was summarized on 
separate NAEP reading scales (one for each purpose), which range from 0 to 500. In 
addition, an overall reading scale, reflecting combined performance in the two purposes for 
reading, was also developed. The overall reading scale also ranges from 0 to 500. 



14 Judith A. Langer, Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our 
Nation’s Schools. (Princeton, NJ: National Assessment of Educational Progress, Educational Testing 
Service, 1990). 

15 The eighth- and twelfth-grade national NAEP reading assessments also measured a third purpose for reading 
- reading to perform a task. 
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This part of the report contains two chapters that describe the reading proficiency of 
fourth-grade public- school students in West Virginia. Chapter 1 compares the overall 
reading performance of the students in West Virginia to students in the Southeast region 
and the nation. It also presents the students’ average proficiency for the two purposes for 
reading. Chapter 2 summarizes the students’ overall reading performance, as well as 
performance for each of the two reading purposes, for subpopulations defined by 
race /ethnicity, type of community, parents’ education level, and gender. 
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CHAPTER 1 

Students’ Reading Performance 

Reading achievement is central to one of the goals adopted by the president and the 
governors following the historic Charlottesville conference — American students will leave 
grades four, eight, and twelve having demonstrated competency in challenging subject matter 
including English, mathematics, science, history, and geography; and every school in America 
will ensure that all students learn to use their minds well, so they may be prepared for 
responsible citizenship, further learning, and productive employment in our modern 
economy}* Concern about attaining the goal and, more importantly, about the reading 
abilities of our nation’s students has increased recently because it appears that many 
students of all ages have difficulty reading thoughtfully. 17 

Reading for meaning involves a dynamic, complex interaction between and among the 
reader, the text, and the context. Readers, for example, bring to the process their prior 
knowledge about the topic, their reasons for reading it, their individual reading skills and 
strategies, and their understanding of differences in text structures. 

The texts used in the reading assessment are representative of common reading demands. 
Students in grade 4 are asked to respond to literary and informational texts which differ in 
structure, organization, and features. Literary texts include short stories, poems, and plays 
that engage the reader in a variety of ways, not the least of which is reading for fun. 
Informational texts include selections from textbooks, magazines, encyclopedias, and other 
written sources whose purpose is to increase the reader’s knowledge. 



16 AMERICA 2000: An Education Strategy (Washington, DC: U.S. Department of Education, 1991). 

17 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress . (Washington, DC: National Center for Education Statistics, 1992); Judith A. Langer, 
Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our Nation's Schools. 
(Princeton, NJ: National Assessment of Educational Progress, Educational Testing Service, 1990); Richard 
C. Anderson, Elfrieda H. Hiebert, Judith A. Scott, Ian A. Wilkinson. Becoming a Nation of Readers. 
(Washington, DC: National Institute of Education, 1985). 
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The context of the reading situation includes the purposes for reading that the reader might 
use in building a meaning of the text. For example, in reading for literary experience, 
students may want to see how the author explores or uncovers experiences, or they may 
be looking for vicarious experience through the story’s characters. On the other hand, the 
student’s purpose in reading informational texts may be to learn about a topic (such as the 
Civil War or the oceans) or to search for specific information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

Students are asked to provide the overall or general meaning of the selection. This 
includes summaries, main points, or themes. 

Students are asked to extend the ideas in the text by making inferences and 
connections. This includes making connections between cause and effect, 
analyzing the motives of characters, and drawing conclusions. 

Students are asked to make explicit connections between the ideas in the text and 
their own background knowledge and experiences. This includes comparing story 
characters with themselves or people they know, for example, or indicating 
whether they found a passage useful or interesting. 

Students are asked to consider how the author crafted a text. This includes 
identifying stylistic devices such as mood and tone. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. All students at all levels should be able to respond to reading 
selections from all of these orientations. What varies with students’ developmental and 
achievement levels is the amount of prompting or support needed for response, the 
complexity of the texts to which they can respond, and the sophistication of their answers. 

As shown in Figure 2, the overall average proficiency of fourth-grade public-school 
students from West Virginia on the NAEP reading scale was 217. This proficiency was 
about the same as that of students across the nation (216). 18 There also was a tremendous 
range in student performance as shown by the percentiles of the distribution of reading 
proficiency in West Virginia presented in Table 3. The lowest performing 10 percent of the 
fourth graders from West Virginia had proficiency levels below 172 while the top 10 percent 
had proficiency levels above 258. 



Initial 

Understanding 



Developing an 
Interpretation 



Personal 

Response 



Critical Stance 




18 Differences reported as significant are statistically different at the 95 percent confidence level. This means 
that with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. 




28 



THE 1992 NAEP TRIAL STATE ASSESSMENT 



West Virginia 



FIGURE 2 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency 



THE NATION’S 
REPORT 
CARD 



1992 



Trial state Assessment 




The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 
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TABLE 3 



Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students 



5th 


10th 


25th 


50th 


75th 


90th 


95th 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



158 ( 2.6) 172 ( 2.6) 195 { 1 .6) { 15)^ 240 ^14% ^ |58| 1,|) 

149 ( 3.1) - 163 { 3.7); 188(3.7) 236 ( 25) 256 { 45) 420jf 3-2^ 

152 { 2i0) ' 168 (1.7) 193 ( 1.1) 218( 1.4)- 241 f14) f6tf 



The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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LEVELS OF READING ACHIEVEMENT 

Average proficiency on the NAEP scale provides an overall depiction of students’ reading 
achievement; however, by itself, it does not describe what students know and are able to 
do, nor does it evaluate student performance against a standard. This report next presents 
a set of results based on applying the National Assessment Governing Board’s standards 
to student performance on the reading scale. 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment.” (Pub.L. 297-100, Section 3403 (a)(5)(B)(ii)). To carry out this responsibility, 
NAGB contracted with American College Testing (ACT) to undertake advisory and 
analytic functions that could assist the Board in forming its conclusions as to appropriate 
achievement levels to be used for evaluating the 1992 reading assessment results. 
Achievement levels are mappings of collective judgments about how students should 
perform onto the achievement scale. 19 Boundary points were developed for three 
achievement levels for each grade — Basic, Proficient, and Advanced. Performance at the 
Basic level denotes partial mastery of the knowledge and skills that are fundamental for 
proficient work at the fourth-grade level. The central level, called Proficient, represents 
solid academic performance at the fourth-grade level. Students reaching this level 
demonstrate competency over challenging subject matter and are well prepared for the next 
level of schooling. Achievement at the Advanced level signifies superior performance in the 
fourth grade. 

This report follows NAGB’s policy that achievement levels should be the primary and 
initial method of presenting the results of the 1992 Trial State Assessment. In this report, 
these achievement levels are applied to the 1992 data, showing the proportions of students 
that achieved the three achievement levels. 

Definitions of the three levels of reading achievement are given in Figure 3. Examples of 
items at the achievement levels are provided. The reading passage which accompanies these 
items can be found in Appendix B. It should be noted that constructed-response items 
occur at all levels of reading achievement. 



19 Appendix C briefly describes the process of gathering expert judgments about Basic, Proficient, and 
Advanced performance -- as defined by NAGB policy — on each reading item, combining the various 
judgments on the various items and mapping them onto the scale, and setting the scale score cutpoints for 
reporting purposes based on these levels. . 
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figure 3 | Levels of Reading Achievement at Grade 4 **** 



The following achievement-level descriptions focus on the interaction of the reader, the 
text, and the context. They provide some specific examples of reading behaviors that 
should be familiar to most readers of this document. The specific examples are not 
inclusive; their purpose is to help clarify and differentiate what readers performing at each 
achievement level should be able to do. While a number of other reading achievement 
indicators exist at every level, space and efficiency preclude an exhaustive listing. It should 
also be noted that the achievement levels are cumulative from Basic to Proficient to 
Advanced. One level builds on the previous levels such that knowledge at the Proficient 
level presumes mastery of the Basic level, and knowledge at the Advanced level presumes 
mastery at both the Basic and Proficient levels. 




Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read . When reading texts 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences. 



Specifically, when reading literary text, they should be able to tell what the story is generally about -- providing 
details to support their understanding — and be able to connect aspects of the stories to their own experiences. 

When reading informational text, Basic-level fourth graders should be able to tell what the selection is generally 
about or identify the purpose for reading it; provide details to support their understanding; and connect ideas 
from the text to their background knowledge and experiences. 



SASIC 

.EEVEL 





Fourth-grade students performing at the Proficient level should be able to 


PROFICIENT ; 


demonstrate an overall understanding of the text, providing inferential as well 


LEVEL 


as literal information. When reading text appropriate to fourth grade, they 




should be able to extend the ideas in the text by making inferences, drawing 




conclusions, and making connections to their own experiences. The connection 




between the text and what the student infers should be clear. 



Specifically, when reading literary text, Proficient-level fourth graders should be able to summarize the story, 
draw conclusions about the characters or plot, and recognize relationships such as cause and effect. 

When reading informational text, Proficient-level students should be able to summarize the information and 
identify the author’s intent or purpose. They should be able to draw reasonable conclusions from the text, 
recognize relationships such as cause and effect or similarities and differences, and identify the meaning of the 
selection’s key concepts. 



ADVANCED 

LEVEL 

(275) 



Fourth-grade students performing at the Advanced level should be able to 
generalize about topics in the reading selection and demonstrate an awareness 
of how authors compose and use literary devices. When reading text 
appropriate to fourth grade, they should be able to judge texts critically and, in 
general, give thorough answers that indicate careful thought. 



Specifically, when reading literary text, Advanced-level students should be able to make generalizations about 
the point of the story and extend its meaning by integrating personal and other reading experiences with the 
ideas suggested by the text. They should be able to identify literary devices such as figurative language. 

When reading informational text, Advanced-level fourth graders should be able to explain the author’s intent by 
using supporting material from the text. They should be able to make critical judgments of the text (including 
its form and content) and explain their judgments clearly. 
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FIGURE 3 
(continued) 



Levels of Reading Achievement at Grade 4 




Trial State Assessment 



The three items that follow were selected to exemplify each of the three achievement levels 
at grade 4. These items are all based on the story “Sybil Sounds the Alarm,” which is 
shown in its entirety in Appendix B. This is an historical narrative demonstrating the 
purpose “reading for literary experience.” For the multiple-choice items, the correct answer 
is marked with an asterisk. For the short constructed-response item, the scoring guide is 
provided. Also shown is the percent correct (conditional p-value) for the students 
performing within the interval of the indicated level. 




Sybil’s father thought that she 

A. was obedient but forgetful 
*B. was courageous and a good rider 

C. could lead the troops against the British 

D. could easily become angry 
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PROFICIENT LEVEL 



Percent Correct for Proficient Interval 



Nation 






The information about the statue and stamp helps to show that 
*A. people today recognize and respect Sybil’s bravery 

B. people were surprised that George Washington honored her 

C. the author included minor details 

D. heroes are honored more now than they were then 
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